A Practical Guide to Testing the Understandability of Notations

Model-driven development is the process of creating models of a software system and transforming them into source code. Since the stepwise transformations can be done automatically or by hand, the notations of the models should be both precise and understandable. This is especially important if the software system is developed by a large, international team where the persons who model differ from the ones who implement the source code based on the models’ content. Understandability and precision can be experimentally tested. This paper presents a guideline for planning and conducting such experiments. The guideline is derived from a theoretical framework and designed to yield valid and statistically significant results by a simple experimental procedure. Additionally, an open-source tool is provided that supports the suggestions. Guideline and tool have been successfully applied in an industrial context: Experiments revealed that a graphical notation used for model-driven development within SAP AG is as precise as a textual notation, but more difficult to understand.